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ABSTRACT 

The Regional Center for Educational Innovation and 
Technology (INNOTECH) of the South East Asian Ministers of Education 
Organization (SEAMEO) evaluates its self-instructional program for 
English listening comprehension in this final report. The program was 
designed fo.r students whose English language proficiency does not 
enable them to follow courses at INNOTECH, which are conducted in 
English, and it was intended both to develop their listening skills 
in technical English and to expose them to part of the actual 
INNOTECH curriculum. The course consists of 50 units recorded on 
cassettes, each with an oral passage of 200-800 words lasting from 
two to eight minutes. The tapes are supplemented by printed material 
which includes questions, answer keys, transcriptions of the texts, 
and step-by-step instructions for the program. The course was found 
to be effective in improving the aural comprehension of technical 
English at least as much as the former program which involved five 
months of classroom instruction. In addition, the report emphasizes 
its economical aspects; programmed materials and cassettes are used 
rather than teacher time and more expensive recording equipment. 
Included with the report are two appendixes; one gives the 
introduction and instructions from the program material and the other 
gives samples of the printed program material. (HW) 
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The TECH Program; Technical Enp;lish Cmprehension (Hearinis:) 
The Aim 

The initial aim in developing the TECH Program was to provide an economical 
means to train participants in Innotech's Training Programs in the comprehen- 
sion of spoken technical English. A secondary aim, introduced during the 
development of the program was to provide an introduction to the content of 
the Innotech curriculum. 

The Problem 

Although English is the announced medium of instruction in Innotech training 
programs, a significant proportion of the participants fail to gain full 
"benefit from these programs because of inadequate proficiency in English. 
Although all participants from the non-English speaking SEAMEO countries have 
had sufficient .training in English to converse in that language when they 
first came to Innotech, a high proportion have difficulty in following 
lectures or in taking part in discussions at the required technical level. 
For approxiiaately half of the trainees test scores indicate that further 
training in English is necessary before participation in programs such as 
Innotech *s can be fully effective. A similar problem is encountered in all 
of the SEAMEO Centers, and the problem is, of course, world-wide: it is 
common to thousands of students who go abroad to study in a language not 
their own. 

The problem vms discussed at length at the 7th SEA?^.C Conference held in 
Vientiane in January 1972, and it was recommended that all- of the SEAMEO 
Centers should provide, "for those participants who need it, an intensive 
2 week course in English before the commencement of the training/ 
rei^earch programme, as well as arrange for the tutoring in English of such 
participants during the first two weeks of their courses." Further, SFAMES 
was instructed "to explore the feasibility of providing funds to meet the 
costs for board and lodging incurred in the two-week intensive courses in 



gp^^" English for participants at the Centers*" 



For a number of reasons the proposal was not feasible. Funds were not 
available to meet the costs of holding the proposed training programs at 
the Centers, qualified instructors were not available at some Centers, 
and experts questioned the effectiveness of a course lasting only two 
Weeks. One alternative, to utilize existing language centers to train 
participants before they departed from their home countries, had other 
disadvantages* Few training courses of the necessary type and level were 
offered; schedules did not match those of the Centers and some of the 
courses required as much as 9 months full time; few courses were available 
outside the capital cities and, of course, many of the participants come 
from outlying areas, 

A feasible alternative appeared to be a self-instructional program which 
would minimize staff requirements and which could be administered anywhere 
at the convenience of the user. The fact that a suitable program was not 
available readymade was not necessarily a disadvantage: a program developed 
locally to solve the immediate problem could be tailored to the special 
needs of the participants and the Center involved. In particular the content 
of the program could be designed to combine an introduction to the Center's 
curriculum with the required training in English comprehension. 

In consequence the Tech Program was designed to provide: 

a. self-instruction in comprehension of oral technical English 

b. an introduction to material in the Innotech curriculum 

c. portability and 

d. economy 

DeveloTanent of t^.e Pro-am 

The program was developed in three phases, each involving a significant 
revision of the procedural and content programs. 

Phase I: April-June, 1972. Small scale tryout of a preliminary form of the 
program. 
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Exploratory work was "begun as a training project carried out by eight of 
the 1971-1972 group of Innotech interns. Considerable motivation was 
provided by one of the interns, Mr. Tea Meng Tech, Inspector of Primary 
School, Directorate of Instruction, Khmer Republic. Mr. Tech had previous- 
ly developed a similar program for his own self-instruction based on radio 
broadcasts in standard and Special English. 

The initial work of the interns produced a preliminary operational program 
(specification of the learning procedure) and a format for the teaching 
materials. The materials for each lesson or unit consisted of a short talk 
and oral questions concerning it reproduced on a cassette tape, together 
with a booklet containing the same questions in printed form, a printed 
transcript of the talk, and instructions specifying the step-by-step 
procedure. On the basis of a tryout In which the interns served as 
experimental subjects the operational program was simplified. The tryout 
also indicated that initial estimates of the appropriate length for 
recorded passages and the difficulty level 01^ the questions were too high. 

Phase II: September 1972-February 1973. Development and tryout of the 
revised operational program and a 50-unit content program. 

On the basis of the initial tryout and other con;^iderations the operational 
program was simplified and a 50-unit content program (syllabus) and the 
material required to teach it were written. The new program was givcm to 
15 of the 16 members of the 1972-73 group of interns beginning in 
Septeanter, short3.y after their arrival at Innotech. (One member of 
the group who haa had almost no training in English before coming to 
Innotech declined.) The 50 units, each of which required a maximum of 
1 fcc?a:'s work^ vere completed in approximately 90 working days over a 
month period. 

A lumber of changes were introduced in the program during this tryotit. 
The full set of program materials was not in final form before the tryout 
began; materials for later programs were being selected and questions 
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written as eairlier units were -being "administered. The performance and 
comments of the lesirners together with experience gained during the 
writing of questions indicated that 8 good questions which met the criteria 
for length and technical level could not be derived from the available 
texts. Consequently J after tiie 20th unit the number of both essay and 
multiple choice questions per unit was reduced from 8 to 5. 

Initially the program included oral questions recorded on the tape, with 
the same questions repeated in printed form. After experience with a 
few units the oral questions were eliminated together with the procedural 
steps which utilized them. The operational program was also .simplified 
in other minor ways. 

Phase III: February- April, 1973. Final Revision. 

This phase was bsgun after all interns had completed the 50 tjnits* 
The final revision was based on a detailed critical analysis of the entire 
program. The performance of the English-proficient participants provided 
useful information concerning the difficulty of questions and text content. 
Revisions include: 

(a) re-ordering of the units 

(b) elimination of 3 unsatisfactory units and the inclusion of 
3 units specially written to provide simpler material at 
the beginning of the program 

(c) reduction of the nmber of questions from 8 to 5 in all units 

(d) revision of the instructions 

(e) revision of the printed format 

(f) editing of questions and texts and 

(g) elimination of discrepancies, mostly typographical, between 
the oral and printed texts. 

After elimination of oral questions, no major changes in the operational 
program were found necessaxy, 

ERIC 



The Program 

The program which resulted from this process of aevelopment consists of 50 
Txnits of material in the areas of educational development in Southeast Asia, 
the systems approach, innovation and educational technology. 

The texts were selected from material in the Innotech file and library, 
including technical articles, seminar reports, etc. 

The criteria used to select the material were: 
(a) Coherence and unity 

("b) Suitability for the development of questions on content 

(c) Length - 2-8minutes3 200-800 words 

(d) Relevance to Innotech training program 

The texts were recorded orally on tape in a variety of voices of native 
English speakers, men and women, with British and American accents # They 
were read so as to simulate a seminar or lecture presentation but rather 
more slowly and distinctly. 

The material for each unit consists of a short oral passage recorded on a 
cassette tape, supplemented by printed material which includes questions, 
answer keys, a transcript of the recorded text and step-by-step instructions. 
Answers to the questions are recorded on expendable answer sheets, one 
required per unit. The format was especially planned to fit the requirements 
of the operational program. 

The overall program is conveniently described separately under two headings, 
which represent those aspects of the program which control the how and the 
what of learning. The operational program specifies how — it is the 
procedure which the learner follows in completing each unit of material. 
The content program is the sequence of learning material and tasks which 
largely determines what the student learns. Details of the two types of 
program follow: 



A. Operational program (see Figure l) 

The operational program, which is the same for each unit, is essentially 
a study guide in the form of a seven- step branched program on the brigh- 
tening model. The brightening model begins with a test, which presents 
the task to be performed in its most difficult form. If the learner can 
not perform the task it is made progressively easier by repeating or 
elaborating the instructions, presenting the task in a simpler form, by 
providing additional information, etc., until the learner succeeds. At 
this point the learner is presented with another task which is similarly 
simplified if he can not perform it. The brightening model, also known 
as the "discovery method" is the inverse of the fading model and the 
conventional teach-then-test classroom, procedures which favor lear-ning 
by rote rather than by problem-solving. 

In each unit of the Tech Program the initial task is a relatively realistic 
representation of "Bie situation the student is being trained to face — he 
listens to a single presentation of oral material presented onceji after 
which he is required to demonstrate that he understood what he heard by 
answering essay-type questions about it. If he fails, the task is 
progressively simplified by allowing him to listen again, by substituting 
multiple choice questions and by allowing him to read a transcript of 
the oral material. The procedural program which specifies the steps in 
this process in detail is surcmarized in Figure 1. 

B. The content program (see Fig\ire 2) 

The content of the 50 units is suggested by thv^ir titles, which are 
listed in Figure 2. In addition to the recorded selection, each unit 
Includes a set of 5 essay questions, 5 multiple-choice questions, answer 
keys and the printed text of the recording, all imbedded in the instruc- 
tions which specify the operational programe 

Texts were selected from material in the Innotech file and library, 
chiefly technical articles and reports given at seminars and other 
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, Figiire 1 

The Operational Prog:ram: 

Siumary Instructions for Each Unit in the TECH Prograni 



STEP 1: Listen to the tape once, without interruption. 
Take notes if you vish. 

STEP 2: Answer the 5 essay questions (QA). 

STEP 3: Listen to the tape again, several times if necessary, and 
revise or complete your answers to QA.. 

STEP k: Mark unacceptable answers to QA, using the answer key. 

If you made no more than one error, go to the next unit. 
If you made two or more errors, go to STEP 5- - 

STEP 5: Answer the 5 multiple choice questions (QM). 

STEP 6: Mark incorrect answers to QM, using the answer key. 

If you mad-e no more than one error, go to the next unit. 
If you made two or more errors, go to STEP 7. 

STEP 7* Read the text as you listen to the tape again. 

Use an English and/or dual- language dictionary if necessary. 
Find the answers to QA and QM questions that were marked 
incorrect. 

On your answer she^^t beside each ciestion missed write the 
. numhers of the lines in the text on which you fotu^d the sjiswer 
to it. 

EI7D 
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Figure 2 
The Content Program 
List of TECH Program Units 

Introductory Units 

Unit 1: Training Unit: How to Be a Better Minister of Education 

Unit 2: Educational Technologies 

Unit 3: Teaching Technologies and Educational Systems 

Educational Development and Innovation 
Unit h: A New Approach 

Units 5 & 6: Educational Development 5.n Southeast Asia: Methods of 
Change . 

Systems Approach 

Units T & 8: Introduction to the Systems Approach 
Urit 9" The Systems Approach to Education 

Instructional Objectives 

Units 10-12: Instructional Objectives 
Alternatives 

Unit 13: Alternatives in Education 

Educational Technology 

Units 1^1 & 15: Educational Technology 

Educational Television 

Units l6 - 18; VJhat the Research on ETV Says 

Units 19-22: Television Reconsidered 

Units 23-25: Review of Schramm^TV Reconsidered" 

Unit 26: Educational Television in Singapore 

Programmed Learning 

Units 27 - 3^: Characteristics of Programmed. Learning 
Unit 35 J The Pacific Horizons Reading Scheme 

Evaluation 

Units 36 & 37: Introduction to »!Evaluation in The Systems Approach" 
Units 38 - i*3: Evaluating Training and Development Systems 

Objectives 

Units - 1*7: Defining Objectives for Six Varieties of Learning 
Unit kQ: Objectives-Based Accountability Procedures for 

Classroom Use. 

Q Teaching Principles 

ERJC Units i+9 - 50: Aptitudes or Specific Skills? 
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meetings. These were recorded orally on tape in eight voices of native 
English speakers, "both men and women, with British and American accents. 
They were read so as to simulate a serainar or lecture presentation hut 
rather more slowly and distinctly. The criteria used to select the 
material were: 

a. Subject matter area ; The area defined hy the curriculum of Innotech 
training programs and topics in closely related fields, esj;ecially 
topics of general interest. 

h. Subject matter level ! Introductory tertiary, suited to interested non- 
specialists such as university-level students entering the field or 
professionals in related fields. 

c. Language level : Tertiary — Assume secondary-level training in English 
and a relatively large reading vocabulary. Suited to the (Educated 
non-specialist: technical terras and concepts are introduced, explained 
and defined in non-technical language. 

d. Coherence and unity : The material should be complete and understand- 
able in itself 5 with minimal dependence on special knowledge or 
context from previous units. Units may, however, be parts of a larger 
whole such as that represented by a series of units on the same or 
related topics . 

e. Content : Sufficient material and specificity to permit the formulation 
of clear and answerable questions, 5 essay-type and 5 multiple-choice, 
with duplication of content in the two types permitted. 

f. Length t 2-8 minutes of oral presentation (200-800 words). 

Questions were constructed with Tie aim of establishing useful listening 
and study habits. Although questions on specific detail were not excluded, 
comprehension rather than memorization was emphasized. In addition to 
reproduction of detail, questions were included systematically to require 
comprehension of the following kinds: 



- 10 



a. Ident if T cation of the main theme or thesis of the text or the author ^-s 
aim, c :ude, bias or emphasis. (Care was taken that questions of 
this type could not be answered from the title of the unit. In many 
cases on obviously descriptive title was replaced by a general or 
noncommittal one to avoid giving? away an answer to this type of 
question. ) 

b. Analysis of hierarchies aiid outline. 

1. Listing of major topics or sub-headings 
2» Distinguishing main and sub-topics 
3* Relating major points to illustrations and examples 
1+. Giving examples and applications of generalizations, from within 
or outside the text 

c. Id-?ntif ication of cause and effect relationships 

d. Interpretation: 

1. Summarization 

2. Identification of wori -meanings from context 

3. Classification 

1+. Comparison (similarities, differences) 
5. Illustration, exemplification 
6» Explanation 

e. Translation (of words, idioms, phrases, sentences) into equivalent 
English forms. 

1. Recognition and production of synonyms, equivalent terms, phrase?, 
etc. 

2. Definition 

Evalxiation 

Evaluation of the program is based primarily on the performance of an 
experimental group of 15 Innotech interns who completed the 50 units in 
the l+~month period between September 27, 1972 and January 23, 1973. The 
pre- and post-test performance of this group on the Michigan Test of 
Aural Comprehension was compared with that of a control group consisting 
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of 13 of the interns from the previous year who had "been given the same 
tests on approximately the same dates and stages of their training . 

Although the program was designed for use "by persons with a deficiency in 

English, the validation groups included those with no deficiency. Scores 

for the experimental group on the Michigan Test, administered during the 

week preceding the start of the program ranged from 20% to 9^%. Of these j 

scores for. nine in"?;erns were "below 80;^, indicating, according to the test 

manual, "considerable handicap in understanding spoken English." Scores . 

for seven of the nine were below 70^, which the manual interprets as follows: 

Students in this range are not roady to undertake academic work 
in an environment where English is the medium of instruction. 
Those at the upper lir:it of this range may know enough English 
to travel, hut they will not ordineirily be able to go into 
academic work until they have devoted some time exclusively 
to the study of English 

Effectiveness of the program was evaluated in terms of (a) improvement in 
the performance of the experimental grouP on the 50 program units, (b) gains 
on the Michigan Test compared with gains achieved by the control group and 
(c) reactions to the program of interns in the experimental group. 

a. Performance on the program units is evaluated in terms of errors on 
essay questions only. Results obtained for multiple choice questions 
were \':ot meaningful since the nambsr of subjects responding to the&e 
questions was not constant. Multiple-choice questions are answered 
in the course of the program only if the leapniars make two or more 
errors on the essay questions 

The number of errors shows a progr^^ssive decrease throughout the program 
(see Figure 3). l^ich of the variability from unit to unit may be 
attributed to differences in the difficulty level of unit content or 
questions. 

There is some evidence that the amouiit of improvement indicated by the 
general trend of the data in Figure 3 is under-estimated. Units 6 and U6, 
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Progress Indicated by Per Cent Error on Essay Questions 
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starred in Figure 3, were given under special test conditions to insure 
comparability. Early in the program it was discovered that some of the 
participants were deviating from the instructions. In order to increase 
their scores on what some persisted in viewing as an examination rather 
than a teaching device, some of the participants habitually stopped 
the tape to provide additional time for note-taking » listened to the 
tape several times or even r^eferred to the answer key before answering 
the essay questions • To demonstrate the effects of these practices, 
the essay questions of Unit 6 were presented to the participants as a 
group under the conditions prescribed by the program, i.e., they were 
required to answer the questions after a single uninterrupted presenta- 
tion of the recorded passage. As Figure 3 shows, when this was done the 
percentage of errors increased markedly, from approximately 20% on Units 
1-5 to 30% on Unit. 6. On Unit U6| administered three months later under 
similar test conditions the proportion of errors was 15% and a comparable 
increase did not occur. The improvement indicated by the relative 
performance on the two controlled tests is considerably greater than 
that indicated by the scores on other units obtained \inder less controlled 
conditions. 

b. The second form of evaluation is based on pre- to post-test gains on 

the Michigan Test of Aural Comprehension. One form of the test was 

given to both groups when each first came to Innotech and a second form. 

\ras given approximately k months later, in February. The experimental 

group (1972-73) canpleted 50 units of the TECH Program in the interval 

between the two tests. In the control group of 13 interns (1971-72) 

seven interns with protest scores below 70 (indicating a serious 

deficiency in English) were given a special English-language training 

program. This program is summarized in the AIR ^'Report of the Technical 

Advisory Services Provided to Innotech", January, 1972, p. 8: 

Innotech attempted to solve the problem by holding sessions for 
the complete group (of l6 interns) in the morning and then 
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splitting the group in the afternoon. The "English-training" 
group (seven Interns) reviewed the morning sessions and 
vorked on English-training exercises developed from basic 
Innotech reading while the "English-proficient group" began 
work on projects. 

... one staff member was occupied full-time with the English 
program • 

Pre- to post-test gains are shown in Table 1, (^a) for the total groups 
and (b) for those subjects whose English was deficient as indicated 
by pre-test scores below TO. In both comparisons gains were greater 

Table 1 

Pre to post-test gains on tht^ Michigan 
Atiral English Comprehension Test 

Experimental (1972-73) Control (1971-72) Diff . 

Pre Post Gain N Pre Post Gain Diff-. 

69.7 75.6 5.9 13 73.1 77.2 k.l 1.8 

50.3 65.7 15.1* 6; 58.7 65*9 7.2 8.2 

for the experimental group » but gains for the total groups differed 
only slightly. For the English-deficient > gains for the Tech-trained 
group were nearly twice those for the control group. However 5 the 
number of cases was small and the difference was not statistically 
significant. 

. Less objective evidence is provided by the attitudes of the participants 
in the program whose English was deficient. With the exception of 
two who appeared not sufficiently motivated for self -instruction 
their attitudes were favoxirable. This subjective evaluation of 
attitudes is supported by twelve requests from persons who have 
purchased the necessary cassette tapes to have copies of the program 
made for their own use. Others whose English was initially adequate 
have requested copies of the printed texts and questions. 



N 

Total 
Group 

English rj 
deficient 
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Conclusions 

The available data do not provide conclusive evidence for the effectiveness 
of the TECH Program. Perf ormance on the program itself showed progressive 
improvemeit but only part of it can be attributed to the program, since 
the learners were simultaneously participating almost full-time in an English- 
language environment. The relatively large difference in favor of the 
TECH Program group in the experimental-control comparison is more convincing, 
but the number of cases was too small for the difference to be significant. 
A more extensive validation study is indicated. However, in both cases 
the data are consistent with the conclusion that the program improves the 
comprehension of technical English. It was at least as effective as a 
conventional program used at Innotech in the previoaxs year. The latter 
program required classroom space and occupied the time of one professional 
staff member- for a period of five months . To administer the TECH Program 
requires the part-time services of a clerk and minimal space for storage. 
The program has high face validity and in the Judgment of many participants 
it was valuable for learning both language skills and subject matter. 

Thus, there are a number of indications that the program is an effective 
aid to learning English comprehension skills and curriculum content, ancl 
that it is at least as effective for achieving these objectives as a 
conventional program which had greater manpower and space requirements 
and is less portable. 



APPENDICES 

I. Introduction smd instructions from the program material. 
II. Samples of the printed program material (Programs 5 and 36). 
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Introduction and instructions from the program material 



ERIC 



CONTFNTS 



INTRODUCTION 
MATERIALS 

TECH PROGRAM ANSWER SHEET 
PROGRAM SUMMARY 
DETAILED PROCEDURES 
PROGRAM UNITS 

I. Introductory Units 

Unit 1: Training Unit: Hov to be a Better Minister 

of Education 
Unit 2: Educational Technologies 

Unit 3: Teaching Technologies and Educational Systems 

II. Educational Development and Innovation 
Unit h: A New Approach 

Unit 5: Educational Development in Southeast Asia* 

Methods of Changes, Part I 
Unit 6: Educational Development in Southeast Asia* 

Methods of Changes, Part II 

III. Systems Approach 

Unit 7: Introduction to the Systems Approach, Part I 
Unit 8: Introduction to the Systems Approach, Part II 
Unit 9' The Systems Approach to Education 

IV. Instructional Objectives 

Unit 10: Instructional Objectives, Part I 
Unit 11: Instructional Objectives, Part II 
Unit 12: Instructional Objectives, Part III 

V. Alternatives 

Unit 13: Alternatives in Education 



VI. Educational Technology 

Unit ll+: Educational Technology, Part I 91 
Unit 15: Educational Technology, Part II 97 

VII. Educational Television 



Unit 


16: 


What the Reseaxch on ETV gays. Part I 


105 


Unit 


17: 


What the Research on ETV says. Part II 


111 


Unit 


18: 


What the Research on ETV says. Part III. 








Which Medium? 


117 


Unit 


19: 


Television Reconsidered, Part I 


103 
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20: 


Television Reconsidered, Part II 


129 


Unit 


21: 


Television Reconsidered, Part III 


135 
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22: 


Television Reconsidered, Part IV 


lUl 
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23: 


Report on MIC Conference (Review of Schrtwrn: 








TV Reconsidered) Part I 


1*7 
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2k: 


Report on AMIC Conference (Review of Schranm: 








TV Reconsidered) , Part II 


153 


Unit 
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Report on AMIC Conference (Review of Schramm: 








TV Reconsidered), Part III 


159 



Unit 26: Educational Television in Singapore l65 
VIII. Programed Learning 
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Part 
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II 
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A Basic Principle of Learning 
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What is Programed Learning ? 
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A Feature of Programed Learning 
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32: 


Linear and Skip Linear Programs 
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33: 


Branching Programs 
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Programed Learning: Books or Machines? 
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35: 
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Introducti on 



TSCH is a self-instructional prograiD' to improve the understanding 
of technical English. It is not interxded for beginners; it is designed 
for those who already read, speak and understand English v/ell enough 
to carry on a conversation and to find their way around in an English- 
spealdLng coimtryj, but ^^ho do not understand spoken English well enough 
to follow technical lectures and discussions. The name TECH is an 
acronym for ^Technical English Comprehension (Hearing). The letter T^ 
could just as v/ell stand for Tertiary, to indicate that TECH is a 
university-level training program for those who have completed their 
training in English at the secondary level. 

The program includes 5C units of material, each of which consists 
of a short talk on a cassette tape, a printed transcript of the talk, 
i Vissay and multiple choice questions, answer keys and a set of instruc- 
tions which indicate ho\r to use this material most effectively. The 
content of the talks is related to the curriculum of Innotech training 
programs s the topics discussed include educational development in 
Southeast Asia, innovations in education, the ^stems approach, 
evaluation and educational technologies such as ETV and programed 
instruction. 

To make the best use of tliis material the program should be 
followed exactly . This program or study guide has been designed 
ver-y carefully on the basis of tested principles of skill-learning 
and it has been tried out and modified repeatedly to improve its 
convenience smd effectiveness for developing the skill of imder- 
standing technical English as it is spoken by native speakers. 
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Materials 

Materials required for the TECH program include: 

1. A standard cassette tape player. The recording feature 
that most tape players include is not used for tlie TECH 
program. 

2. 25 TECH program cassettes, each of which presents the 
recorded niaterial for two units, one imit on each side. 
The recorded material for each unit is a short talk, 2 
to 8 minutes in length. 

5. The TECH BOOK, which contains, in addition to the intro- 
duction you are now readings the printed material for 
50 TECH units. This includes, for each imit, 5 essay 
questior^s and 5 multiple choice questions iri.th answer 
keys i'or each and a printed, transcript of the recorded 
material, 

4. 50 ansT^er sheets, one for each \m±t (see next page for 
the format of the answer sheet). If answer sheets are 
not available blank sheets may be used.' In that case it 
is coni'enient to arrange the ansvrers as on the answer 
sheets, leaving the necessary space for narking wrong 
ansv/ers azid the numbers of the lines in the text v/here 
the ansvrers may be found. 

5* Pen or pencil and paper for taking notes or* the recorded 
talks. 
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TBCH Program Answer Shaef 
Name Date Unit 
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Revision 



2. 



Revision 



3. 



Revision 



4. 



Revision 



5. 



Revision 



QA ilnswers 



OA 
Lines 



QI4 
Ans. 



I ! 



Lines 
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Program Summ ary 

There are seven steps in the program: 

Suiaiaa ry Instruct ions for Bach Unit i n the TECH Program 

STEP It Listen to the tape once, without interruption. 
Take notea if you wish, 

Si'EP 2t Ansuer the 5 essay ciuestions (^U)* 

STEP 3r Licteu to th.3 tape agcun, several times if necessary, 
and revise or complete your answers to ^i. 

STEP 42 Hark unacceptable ansur^rs to ciL^. using the answer kay. 

If you made no more than one error > go to the next unit* 
. If you made tuo or more errors, go to STEP 5* 

STEP 5: Answer the 5 multiple choice questions (OW) . 

STEP 6: Hark your answers to QI'1> using the answer key. 

If you made no more than one error, go to the next imit. 
If you made two or more errors, go to STEP ?• 

STEP 7s Read the text as you listen to the tape again. Use an 
English and/or dual-* language dictionary if necessary. 
Find the answers to and 0J3 questions that were 
marked wrong. On your ansvjrer sheet beside each ques- 
tion missed, vrrice the number^i of the lines in the 
text on which you found the answer. 

More specific instructions and eirplanations for each step follow. 
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Detailed Procedures 

Step 1: Listen to the tape once, without interruption. 
Take notes if you wish. 

Prepare to answer questions concerning the general 
purpose and major points made by the speaker as well as 
questions on specific detailo. 

Do rot stop the tape or listen to it a second time before 
ansi'jering questions in Step 2. 

Step 1 begins the program vdth a realistic presentation 
of the situation you vrill face at Innotech or in any 
academic or training program where instruction is 
conducted in English - you will be expected to understand 
lectures or discussions that you can hear only once. 
In the real situation you can taJce notes, but you cannot 
slow down the speaker or expect him to stop and wait 
while you take notes and you can not expect him to repeat 
what he says for your benefit. This is the situation 
reproduced in Step 1. After listening to the talk once 
as Step 1 requires you will very likely find that you 
do not virderstand it fully; yoii vail probably not be able 
to answer all of the questions satisfactorily when you 
reach Step 2. This does not mean th^t you should listen 
to the tape more than once or stop the tape to take notes. 

If you did this you could probably ans;/er more questions 
correctly on Step 2, but you would not be learning to 
understand spoken English under realistic conditions. 
If your purpose in taking the TECn program were to get 
a high scoi^e or to memorize ansivers to the questions, you 
could do muoh better by skipping Step 1 entirely, for 



example, by looking at the answer key as in Step 4 or 
by reading the text as in Step 7 before answering the 
questions in Step 2. But this is not your purpose in 
completing the TECH program - your purpose is to learn 
to understand technical English when you first hear it. 
To learn this skill you must not take short-cuts which 
enable you to answer questions without practicing the 
skill itself. For this reason you should follow the 
instructions for the program exactly. 



Step 2: Answer the 5 essay questions (qa) . 

Ansv/er each question in tlie first two lines provided in 
the answer sheet. The second two lines marked "Revision" 
are to be used for changes or additions to your answers 
which you may wish to make later after listening to the 
tape again in Step 3. 

Do not try to answer in complete sentences. Your answers 
should be very brief, one or two words or a phrase. If 
you require more than two (or even one) lines to express 
your meaning you have probably missed the point. 
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Step 3- Listen to the tape again, several times if necessary, and 
revise or complete your answers to 'iA. 

V/rite your corrected ansvrers on the lines marked 
"Revision"on the answer sheet. 

Step 3 is designed to give you more practice in 
listening to spoken English. The task .this time is 
easier, you may listen several times and, since you 
have read the questions, you Icnow uhat to listen for , 
In the process of revising your answers through listening 
a second or third time you v/ill hear things you missed 
the first time, and understand more. 



Step 4: Mark unacceptable answers to using the answer key, 

If you made no more than one error, go to the next unit. 
If you made two or more errors, go to Step 5. 

Mark omitted or unacceptable answers with an iX, To 
be acceptable your answer need not be in the same words 
as the answer key. Your answer is ac-^eptable if th^ 
words you wrote have the same meaning as the words in 
the answer key. It is not acceptable to say, "Oh, I 
knew the right answer - I just did not v/rite it down 
correctly**. If you did know the answer, why did you 
write something else?) 

Step 4 provides you \vrith "feedback" in the form of 
information that your answers were right or v.Tong, which 
is extremely important for efficient learning. If you 




answered all or all but one of the questions acceptably. 
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you have probatly learned all you cai: learn from the 
task presented. In that case, you should go on to 
the next unit. If not, you need additional v/ork. 
Perhaps you did not understand some of the words or 
idioms, or you could not express jrourself well enough 
to anvTser the e?5say-type questions. 



Step 5: Answer the 5 multiple choices (oj-Or 

V/ritG the letter a, b, c, or d under QI»I Answer on the 
ansv;er sheets 

Step 5 presents a second set of questions in multiple- 
choice form, which are easier than, essay questions since 
they test your understanding vdthout requiring you to 
express yourself in English. 



Step 6: Mark your answers to Ql-i, usijig the ansvzer key. 

If you made no more than one error, go to the next unit. 
If you made two or more errors, go to Step 7, 

Kark \rvong answers with an X* 

Step 6 again provides feedback. If your answers on 
Step 5 indicate that you understood what was presented 
in the talk, you are ready for the next unit. If you 
failed on Step 5 after listening to the talk several 
times in Steps 1 and 3f it is not likely that listening 
again under the same conditions will improve your 
under standin g . 
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Step 7 5 Read the text as you listen to the tape again. Use an 
English cr a dual-lan£,-ua^e dictionary if nec^assary. 
Find the answers to OA and Q}1 r«uestions that were marlced 
vn7ong. On your answer sheet beside each question missed, 
\rrite the numbers of the lines in the text on which you 
found the answers. 

Write the line munbers imder "lines" on the answer sheet. 

In Step 7 you should read the text as you listen to the 
tapes. This will onable yow to identify words that you 
did not recognize or hear correctly and to find their 
meanings. V/ith the aid of the printed text you nay find 
that the speaker did not say what you thought you heard, 
or mean v/hat you thought he meant • 

In this step the tape may be stopped or re-played as 
often as necessary for you to achieve full understanding. 
Identifying the line or lines on which the answers to 
the questions are found provides you with a concrete 
indication that your understanding ia ^^idequate. 



Note: The first time t^iTough each unit you are strongly advised 

to follov: the program exactly. If your purpose is to acquire 
the skill of understanding what you hear, this is the most 
effective ;,)rocedure. But once ydu Lave done this, you may benefit 
by using the program material in other ways as study material: 
for example, for further practice in listening, to gain 
familiarity with the idioms and speech patterns of native English 
speakers, or to study the content of the talks. For these 
Q ' purposes you may develop solf-instructional prograars of your own. 
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Samples of the printed program material (Programs 5 and 36) 
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Unit 5 

Educational Dovelopaent in Southeast Asia 
Methods of Change: Part I 

IMPORTANT NOTES Do NOT look at any questions or answer keys 

until you are told to do so in the instructions 
which follow. 



STEP It Listen to the tape once without interruption. Do not 

stop the tape or play it back# Take notes if you wish. 
When you come to the end of the tape, stop it, and go 
to STEP 2, which you will find on the next page. 
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STEP 2s Answer the questions below (QA), one at a time. Write 
your answers on the answer sheet. If you do not know 
the answer v:o a question, leave the space blank. 
(In cottipleting this step, do HOT look at the answers 
on the opposite page and do NOT look at questions below 
the one you are answering. If necessary^ cover them.) 
When you finish the five questions, go to STEP 3. 



QA2 Essay Questions 

1. How does the speaker describe the general attitude of educators 
to change? 



24. Why should there be a change in the present educational systems 
in Asia? 



3. Name one of the speaker's objections to importing **gadgetry** 
from elsewhere. 



A. Is the speaker opposed to educational technology? 
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5. Who does the writer say should take the initiative for solving 
the educational problems of Asia? 



- 39 - 



Unit 



STEP 3s Listen to the tape again, several times if necessary. 
Complete or revise the answers you gave in STEP 2. 
(Do NOT look at the answer key below, yet,) 
When you have given the best answers you can, go to 
STEP 4t 



STEP 45 Score your answers to QA, using the key below. 
Mark an omitted or ^vTong answer with an Xo 
If you marked no answers wrong, or only one, you have 
finished the unit. Go to the next unit. 
If you marked two or more answers wrong, go to STEP 5. 



QA Answer Key 
1. Conservative OR resistant to change. 



2. Present approaches too costly. 



3» Innovation should be suited to Asian problems OR gadgetry 
is usually added to the cost of the existing system. 



4« No (but he says it must be used properly and economically). 



5« Asians* 
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STEP 5: Answer the multiple-choice. questions (QM) below#on your 
answer sheet. Do NOT look at the answer key on the 
opposite page. 

.When, .you have answered all of the questions, go to STEP 6, 



QMs Multiple Choice Questions 

1. According to the speaker, which of the following is the greatest 
obstacle to change in education? (a) the lack of educators, 
(b) the lack of equipment, (c) the lack of expertise.' (d) the 
conservative attitude of educators. 



2. To solve Asian educational problems, the speaker recoininends 
(a) better funding, (b) educational technology, (c) educational 
research, (d) greater use of foreign experts. 



3. According to the speaker, adding educational technologies to the 
existing educational system will (a) increase costs, (b) decrease 
costs, (c) solve the quantitative problem in education, (d) solve 
the qualitative problem in education. 



4. According to the speaker, the underlying problem for Asian 
education is (a) increasing costs, (b) insufficient teachers« 
(c) poorly trained teachers, (d) all of these. 



5. The speaker believes that Asians may soon be ahead of others in 
educational innovation because (a) Asians are highly adaptable* 
(b) Asians are highly resourceful* (c) the greater need will 
gj^^" ' produce solutions in Asia, (d) the pace can be slower in Asia* 
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STEP 6g Score your answers to QM. ^^ark errors with an X. 

The correct answers ares d, b, a^ it* • 

If you made no errors or only one., you have completed 
the linitt 

If you made two or more errors, go to STEP 7, below. 



STEP 7s Read the text below as you listen to the tape again. 
Read and listen as man]^ times as you wish. 
Find the answers to QA and QM questions that you marked X« 
On your answer sheet, write the numbers of the lines in 
the text on which you found these answers. 



Educational Development in Southeast Asia 
Methods of Changes Part I ' 

I aiT; not foolish enough to propose or to even imply that educational 
innovation and technology can come quickly or easily. The roots of 
the educational establishment lie deep, and this is true in any 
culture — within any national boundary. Educators are generally 
5 conservative and resistant to change. And there are vested interests 
lobbying for maintenance of the status quo. One can see all of these 
forces at work in the countries now coining to grips with the newer 
educational technology. But in Asia the alternative to change is 
perpetuation of "insurmountable" problems, and eventually, quite 

10 possibly, educational bankruptcy, for the cost of traditional 

approaches grows greater each year, and resources for the national 
investment in education, unfortunately, do not increase proportionately. 
There simply must be willingness to try new approaches and there must 
be serious exploration of the newer educational technology to see 

15 what it may have to offer. However difficult the achievement of 
change, a start has to be jnade. 

It would be a mistake, and a serious mistake, to begin simply by 
importing teaching machines, programed textbooks, educational TV, 
computers, and the other "gadgetry" developed for use elsewhere and 

20 attempting to impose them upon an existing system of education. There 
are two things wrong with this approach. First, innovation must be 
suited to Asian problems and needs, if it is to be effective. 
Second, imposing gadgetry upon the existing system simply adds 
the cost of the new to the cost of the old., Educational television, 

25 for example, used as a supplement to traditional classroom instruction 
is a luxury few if any Asian countries can afford. But when used to 
make the skill of the master teacher available to fifty classrooms 
^t a single performance, educational television may become an instru- 
ment for change while effecting genuine economies in the cost of 

30 education. 
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It would also be a mistake to start innovation by in^Dorting specialists 
from other countries to do the job for the Asians. Such expertise 
. must be tapped for purposes of training and for dealing with specific 
problems^ but innovation is not a one-shot affair^ it is a continuous 
• -35 process, and to insure its continuity Asians themselves must be the 
. innovators^ not simply passive onlookers while the job is done for 
them by outsiders. Indeed, there is good reason to believe that once 
the situational pre-requisites for innovation are established, the 
Asitos may easily move into the fore-front in the development and 
AO practical applications of educational technology and themselves become 
the sought-after specialists in this movement. The reason for believ- 
ing- this is simply that there is in Asia" a far more desperate and 
urgent need for more efficiency and greater economy in operating the 
educational establishment thaii there is in the more affluent societies 
where the pace can be slower and the conservative elements can be 
accommodated with more patience* Need breeds effort. Desperate need 
fosters greater effort. 
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Introduction to "Evaluation in the Systems Approach", Part I 
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IMPORTANT NOTE: Do NOT look at any questions or answer keys 

until you are told to do so in the instructions 
which follow. 



STEP 1: Listen to the tape once without interruption* Do not 

stop the tape or play it back* Take notes if you wish. 
When you come to the end of the tape, stop it, and go 
to STEP 2, which you will find on the next page. 
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STEP 2: Answer the questions below (QA), one at a time. Write 
yoiir answers on the answer sheet. If you do not know 
the answer to a question, leave the space blank, 
(in completing this step, do NOT look at the answers 
on the opposite paga and do NOT look at questions below 
the one you are answering. If necessary, cover them.) 
When you have finished the five questions, go to STEP 3. 



QA; Essay Questions 
1. What is the theme of this unit? 



2. List the two major considerations in evaluating new instruc- 
tional materials and procedures. • 



3. How does one select the solution(s) from the list of many 
possible solutions? 



According to the speaker, what is the usual procedure for 
evaluating the effectiveness of new instructional materials? 



5. What does a control group study? 
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STEP 3: Listen to the tape again, severatl tl:n.as if necessary. 
Complete or revise the answers you gave in STEP 2. 
(Do NOT look at the answer key below, yet,) 
When you have given the best answers you can, go to 
STEP k. 



STEP h: Score your answers to QAj using the key below, 
Mark an omitted or wron^g answer with an X, 
If you marked no answers wrong, or only one, you have 
finished the unit. Go to the next unit. 
If you marked two or more answers wrong, go to STEP 5* 



OA Answer Key 

1, Evaluation (of new instructional materials or techniques). 



2, (a) Cost and (b) effectiveness. 



3, "Screening" OR "short-listing" OR choosing one or a few 
solutions that appear to be most cost-effective. 



Test learning with the new materials and compare the performance 
of the experimental group with a control group* 



5 , Conventional ' materials . 
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STEP 5:. Answer the multiple choice questions (QM) below, on your 
answer sheet. Do NOT look at the answer key on the 
opposite page. 

When you have answered all of the questions, go to STEP 6. 



QM: l^ltiple Choice Questions 

1. In this unit the speaker is mainly concerned with (a) evalua- 
tion objectives, (b) evaluation design, (c) evaluation problems, 
(d) evaluation of new instructional materials. 



2. This talk is mainly concerned with the evaluation of 

(a) effectiveness, (b) cost, (c) resources, (d) implementation. 



3. In evaluating new materials, (a) we assess the performance of 
two experimental groups, (b) we assess the performance of two 
control groups, (c) we compare the performance of the 
experimental group with a control group, (d) all of these. 



U. The control group usually studies (a) methods of controlling 
costs, (b) conventional materials, (c) methods of controlling 
materials, (d) new materials. 



5. Evaluation is a procedure for determining (a) usefulness, 
(b) cost, (c) effectiveness, (d) all of these. 
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STEP 6: Score your answers to QM. Mark errors with an X. 

The correct answers are: d, ^, c_, ^, id. 

If you made no errors or only one, you have. completed 
the unit. 

If you made two or more errors , go to STEP 7 , helow. 



STEP 7: Read the text helow as you listen to the tape again. 

Find the answers to QA and QM questions that you marked X, 
On your answer sheet j write the numhers of the lines in 
the text on vhich you found these answers. 



Introduction to "Evaluation in the Systems Approach", Part I 

By now you will have all seen a list of the steps in the systems 
approach to solving educational problems, and you have had some 
practice in starting to solve several prohlems. Let me review 
the steps you have already worked with. In the f".rst step, you 
5 attempted to define the real problem. Next, you ;jrepared the 

"behavioral objectives or outcomes that a solution to this problem 
should produce. Then you thought of as many solutions as possible. 
The rest of the steps in the systems approach are concerned with 
choosing the best solution, and refining, it to make it as 

10 effective as poscible. Up to now we have not talked very much 
about how to chooot. oLo bert solution. When we try to determine 
how useful a solution is, we say that v/e are "evaluating" the 
solution. The procedures that wa use to evaluate solutions are 
"evaluation" procedures. T" is seminar is about evaluation 

15 procedures. That is, it is about procedures that we use to decide 
on ths usefulness of solutions. 

Now, "evaluation" is a very general tern. We may want to evaluate 
different arrangements of office space. We may want to evaluate 
the effectiveness of a newsletter. We niay want to evaluate a new 

20 type of instructional materials. In fact, we should probably 
evaluate any new materials , procedures, programs, etc. that we 
use. Many of the evaluation procedures that we talk about in this 
seminar are very general. They can be applied to many types of 
problems. I suspect, however, that in the beginninig, it will be 

25 easier to understand the procediores if we think of them in 

relation to specific examples. So from now on in this seminar, I 
will talk mainly about evaluating new in struct ionej. materials or 
techniques . In other wor^io., I will talk about procedures to use 
in deciding whether a set of new materials is useful at all, whether 

30 it is better than conventional materials, or whether it is better 
than sane other set of new materials. 
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Let me be even more specific as to what we will cover in this 
seminar. In evaluating new laaterials, we are usually concerned 
with two major questions: 

35 1. How much does it cost? 

. 2. How effective is it in the situation where we want to use 
it? 

Both questions are equally important, but in this seminar we will 
be concerned mainly with how to evaluate the effectiveness of 
1+0 materials. 

I should also maJce it very clear that the procedures we will be 
discussing are complex and time consuming. Because so much time 
and money eire required to do a careful evaluation of even one set 
of materials, it is never possible to apply detailed evaluation 

U5 procedures to all the possible solutions that we can think of. 
Instead, after we have completed our list of possible solutions > 
we "screen", or "short-list" the solutions to select one, (or 
possibly two or three) solutions that appegr to be the best ones 
in terms of cost and effectiveness. In preparing the short-list 

50 we may have to rely on rough estimates, and educated guesses. 

Then we go through our careful evaluation procedtires to get precise 
information on how effective the materials are. 

In general, the procedure for evaluating the effectiveness of new 
materials is thisi we choose a group of sti;idents to study the new 

55 materials, and after they have completed the materials, we give 

them a test to see how much they have learned. Usually we pick one 
or more other groups that study the same subject by conventional 
or alternate methods, so that we can compare the new method with 
the conventional method. The group of students that uses the new 

60 materials is called the "experimental" group. The groups that 
study the conventional materials for comparison are called the 
"control" groups . 



